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ABSTRACT 

The 101 computer-based education (CBE) studies 
considered for use in this meta-analysis came from three major 
sources: references in an earlier meta-analytic review of CBE at the 
college level (J. Kulik, et al., 1980); a computer search of the 
Comprehensive Dissertation Abstracts and ERIC databases; and 
utilization of the bibliographies contained within the documents 
located through reviews and computer searches. The instructional 
outcome measured most often in the 101 studies was student learning 
as indicated on achievement examinations given at the end of a 
program of instruction. Some additional outcome variables measured 
included: performance on a follow-up or retention examination at a 
later date; attitudes toward computers; course completion; and amount 
of time needed for instruction. Findings indicate that computer-based 
education usually has positive effects on college students (CBE 
raised student examination scores by 0.26 standard deviations in the 
average study); CBE effects were somewhat lower in unpublished 
studies than they were in published reports; CBE effects were also 
somewhat lower in the hard, nonlife sciences than in the social and 
life sciences and education; CBE produced small but positive changes 
in student attitudes toward instruction and computers; and CBE also 
reduced substantially the amount of time needed for instruction. A 
12-page reference list and 4 tables complete the document. 
(Author/JB) 
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Abstract 



A meta-analysis of findings from 101 controlled 
evaluations showed that computer-based education (CBE) 
usually has positive effects on college students. CBE 
raised student examination scores by 0.26 standard 
deviations in the average study — a small but significant 
effect. CBE effects were somewhat lower in unpublished 
studies than they were in published ones, and they were also 
somewhat lower in the hard, nonlife sciences than in the 
social and life sciences and education. In addition, CBE 
produced small but positive changes in student attitudes 
toward instruction and computers. Finally, CBE also reduced 
substantially the amount of time needed for instruction. 
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Effectiveness of Computer-Based Education 
In Colleges 



Over the centuries technological advances have had a 
profound impact on education. The development of writing, 
for example, liberated education from oral tradition and 
greatly reduced the need for learners to memorize vast 
quantities of information. The development of printing had 
equally important effects on education. It opened up 
libraries of new information for teachers and students and 
greatly increased educational efficiency. 

Some social commentators are now predicting that 
computer technology will change education in the years ahead 
as completely as the invention of writing and printing did 
in centuries past. Researchers long ago demonstrated that 
computers can wor r i in schools as drill masters, tutors, 
testers, and diagnosticians of educational problems. But 
until recently the cost of computer-based teaching systems 
stood in the way of wide-scale use. With the development of 
small, quick, inexpensive microcomputers during the last 
decade, computing costs have dropped dramatically, and a 
computer revolution in education has become a real 
possibility. 

College teachers have already begun to feel the force 
of the computer's impact. Twenty years ago, computer 
terminals made their way into research laboratories and 
changed the way that college researchers analyzed their 
data. Ten years ago, computers found their v/ay into college 
offices and changed the writing habits of many teachers. 
Today, microcomputers are coming into the classroom, and 
they are changing the way that college teachers teach and 
college students learn. 

The roots of this computer re olution in teaching 
stretch back nearly 30 years to the invention of the 
Skinnerian teaching machine. In his 1954 article "The 
Science of Learning and the Art of Teaching," the 
psychologist B. F. Skinner argued that machines could teach 
more reliably and effectively than human teachers do. They 
could present lessons in a sequence of small steps, wait 
patiently for the learner's response at each step, and 
reinforce each response immediately. Programmed machines, 
Skinner believed, could make teaching more effective and 
learning more joyful. 

A second landmark in the technological revolution came 
a few years later with the development of individualized 
systems of instruction. Like Skinnerian programmed 
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instruction, these individualized systems emphasized 
independent, self-paced work with print materials, but 
individualized systems used longer instructional units — 
often called learning activity packages or modules — that 
gave learners more freedom to choose among different means 
of learning. An especially important feature in these 
systems was the requirement that all learners demonstrate 
their mastery of each unit of material on repe^table unit- 
mastery tests. Individually Prescribed Instruction, Project 
Plan, Individually Guided Education, and Keller's 
Personalized System cf Instruction are probably the best 
known of the individualized systems developed during the 
1960s (J. Kulik, 1983). 

The third stage in this technological revolution was 
marked by the development of computer-based education (CBE). 
In early applications, the computer simply delivered 
programmed instruction and managed individualized teaching 
systems. The marriage of computer technology and programmed 
instruction came to be known as computer-assisted 
instruction (CAI); the marriage of computer technology and 
individualized systems produced computer-managed instruction 
(CMI). More recently, computers have been used for more 
sophisticated teaching jobs. Vhey have served as tools in 
mathematics and writing classes and as simulation devices in 
classes in the natural and social sciences. Some educators 
now argue that students learn most from computers when they 
are used in this way to provide computer-enriched 
instruction (CEI ) . 

The educational revolutions based on writing and 
printing ran their course without any help from educational 
research. No one tried to measure educational outcomes 
while these revolutions were in progress. Scientific tools 
for measuring, predicting, and controlling social events 
were unavailable. The computer revolution is different. It 
is occurring at a time when we have tools for evaluating 
specific programs and tools for drawing general conclusions 
from a collection of specific evaluations. 

These tools have already been used to evaluate CBE 
effects. In a typical evaluation study, a researcher 
divides a class of students into an experimental and a 
control group. Members of the experimental group receive 
part of their instruction at computer terminals, whereas 
students in the control group receive their instruction by 
conventional methods. At the end of the experiment, the 
researcher compares responses of the two groups on a common 
examination or on a course evaluation form. Such evaluation 
studies have been carried out often enough to give some 
indication of the overall worth of CBE in college teaching. 

Reviews designed to integrate the findings from the 
evaluation studies are of two basic types: narrative 
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accounts and meta-analyses. Narrative reviewers usually 
provide concise summaries of major studies and often draw 
conclusions about overall effects based on these studies. 
Reviewers using meta-analysis take a more quantitative 
approach to evaluation results (Glass, McGaw, & Smith, 
1981). They use (a) objective procedures to locate studies; 
(b) quantitative or quasi-quantitative techniques to 
describe study features and outcomes; and (c) statistical 
methods to summarize overall findings and explore 
relationships between study features and outcomes. 

Narrative reviews have seldom reported dramatic 
educational advantages from computer-based instruction at 
higher levels of education. Jamison, Suppes, and Wells 
(1974), for example, reviewed nearly a dozen small-scale 
studies of CBE in college classrooms. Most of these studies 
were carried out in courses operated as part of research and 
development projects in computer-assisted instruction. 
Jamison and his colleagues reported that the results of the 
studies defied easy summary. Computer-assisted instruction 
appeared to be about as effective as traditional 
instruction, they finally concluded, but they also pointed 
out that most alternative methods of instruction appear to 
be about as good as conventional teaching at the college 
level. 

Kulik, Kulik, and Cohen (1980) carried out a major 
meta-analytic synthesis of evaluation findings on CBE at the 
college level. Their review integrated results from 59 
independent evaluations. The meta-analysis showed that CBE 
made small but significant contributions to students' 
academic achievement and also produced positive, but again 
small, effects on student attitudes. In addition, CBE also 
reduced substantially the amount of time needed for 
instruction. In general, Kulik and his colleagues found 
little relationship between study findings and design 
features of evaluations, settings for the evaluation, or 
manner and date of publication of the findings. 

The value of reviews such as these is limited by at 
least two factors. First, the reviews do not cover recent 
applications of the computer in college teaching. None of 
the studies reviewed by Jamison et al. (1974), for example, 
was published after 1972; none of the studies reviewed by 
Kulik et al. (1980) was published after 1978. Computers 
have changed dramatically since that time. They have become 
smaller, less expensive, more reliable, and quicker in their 
operations. Communication with them has become easier, and 
their output has become more readable and attractive. These 
developments have influenced not ^nly the ways in which 
computers are being used in college teaching today, but also 
the subject areas to which they are being applied. 
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A second factor that limits the value of earlier 
reviews is their methodology. Early reviews of CBE 
effectiveness were written at a time when rapid progress was 
being made in the development of a methodology for research 
synthesis. Even the quantitative syntheses, for example, 
did not incorporate the most recent advances in meta- 
analytic methodology. Early users of meta-analysis, for 
example, were often unselective in choosing their studies; 
they often inflated sample sizes by using nonindependent 
findings in a single statistical analysis; and they often 
reported their results in a sketchy fashion (J. Kulik, 
1984). Today's meta-analysts try to avoid these 
methodological flaws. 

This review is meant to supplement earlier reviews on 
the effectiveness of CBE at the college level. It updates 
these reviews and uses currently accepted methods for 
integrating and reporting evaluation findings. The article 
asks questions such as these: How effective 5s CBE at the 
college level? Is it especially effective for certain types 
of outcomes or certain types of students? Under which 
conditions does CBE appear to be most effective? 



Method 

The meta-analytic approach used in this review is 
similar to that described by Glass, McGaw, and Smith (1981). 
Their approach requires a reviewer (a) to locate studies of 
an issue through objective and replicable searches; (b) to 
code the studies for salient features; (c) to code study 
outcomes on a common scale; and (d) to use statistical 
methods to relate study features to outcomes. 

Data Sources 

The studies considered for use in this meta-analysis 
came from three major sources. One large group of studies 
came from the references in our earlier meta-analytic review 
of CBE at the college level (J. Kulik et al., 1980). A 
second group of studies was located by computer-searching 
two library data bases using Lockheed's Dialog Online 
Information Services. The data bases searched in this way 
were (a) Comprehensive Dissertation Abstracts , and (b) ERIC , 
a database on educational materials from the Educational 
Resources Information Center, consisting of the two files 
Research in Education and Current Index to Journals in 
Education . A third group of studies was retrieved by 
branching from bibliographies in the documents located 
through reviews and computer searches. 

These search procedures yielded 101 studies that met 
four basic criteria for inclusion in our data set. First, 
the studies had to take place in actual college classrooms. 
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They had to involve real teaching, not an analog of 
teaching. Second, the studies had to provide quantitative 
results on an outcome variable measured in the same way in 
both a computer-taught and a conventionally instructed 
class. Uncontrolled "experiments" and anecdotal reports 
were not acceptable. Third, the studies had to be free from 
such crippling methodological flaws as (a) substantial 
differences in aptitude of treatment and control groups, (b) 
unfair "teaching" of the criterion test to one of the 
comparison groups, and (c) differential rates of subject 
attrition from the groups being compared. And fourth, the 
studies had to be retrievable from university or college 
libraries by interlibrary loan or from the Educational 
Resources Information Center, the National Technical 
Information Service, or University Microfilms International. 

These standards kept us from using 6 of the 59 reports 
cited in our earlier reviews (1980). (a) One study 
(Ozarowski, 1973) was eliminated because it covered 
nontraditional adult education rather than college teaching, 
(b) Another study was eliminated because it did not include 
results from a conventionally instructed control group 
(Gallagher, 1972). (c) Arsenty and Kief fer's (1971) study 
was eliminated because it did not report results on an 
objectively measured criterion; only teacher assigned grades 
were examined. (d) Three studies were not used in this 
analysis because their report of results was insufficient 
for the calculation of size of effect (Dudley, Elledge, & 
Mukhe^jee, 1974; Hsiao, 1973; and Kromhout, Edwards, & 
Schwarz, 1969). 

Outcome Measures 

The instructional outcome measured most often in the 
101 studies was student learning, as indicated on 
achievement examinations given at the end of the program of 
instruction. Other outcome variables measured in the 
studies were the following: (a) performance on a follow-up 
or retention examination given some time after the 
completion of the program of instruction; (b) attitude 
toward computers; (c) attitude toward instruction; (d) 
attitude toward school subjects; (e) course completion; and 
(f) amount of time needed for instruction. 

For statistical analysis, outcomes had to be expressed 
on a common scale of measurement. The transformation used 
for this purpose was the one recommended by Glass et 
ali (1981). Like Glass and his colleagues, we coded each 
outcome as an Effect Size (ES) , defined as the difference 
between the me^n scores of two groups divided by the 
standard deviation of the control group. For studies that 
reported means and standard deviations for both experimental 
and control groups, ES could be calculated directly from the 
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measurements provided. For less fully reported studies, ES 
could usually be calculated from statistics such as t and F. 

The application of the formulas given by Glass and his 
colleagues was straightforward in most cases. In some 
studies, however, more than one value was available for use 
in the numerator of the formula for ES and more than one 
value was available for the denominator. For example, some 
investigators reported raw-score differences between groups 
as well as covariance-adjusted differences, and some 
reported differences on a post-measure as well as 
differences in pre-post gains. In such cases, we used as 
the numerator of ES the difference that gave the most 
accurate estimate of the true treatment effect. That meant 
using covariance-adjusted differences rather than raw-score 
differences, and differences in gains rather than 
differences on posttests. In addition, some reports 
contained several measures of variation that might be 
considered for use as the denominator of ES. We use the 
measure that provided the best estimate of the unrestricted 
population variation in the criterion variable. 

For measurement of the size of CBE effects on course 
completion, we used the statistic h (Cohen, 1977). This 
statistic is appropriate for use when proportions are being 
compared. It is defined as the difference between the 
arcsine transformation of proportions associated with the 
experimental and control groups. To code CBE effects on 
instructional time, we used a ratio of two measurements: 
the instructional time required by the experimental group 
divided by the instructional time required by the control 
group. 

Study Features 

A total of 17 variables were used to describe 
treatments, methodologies, settings, and publication 
histories of the studies (Table 1). These 17 variables were 
chosen on the basis of (a) an examination of variables used 
to describe study features in previous reviews, and (b) a 
preliminary examination of dimensions of variations in the 
studies located for this analysis. Two coders independently 
coded each of the studies on each of the variables. The 
coders then jointly reviewed their coding forms and 
discussed ar.y disagreements. They resolved these 
disagreements by jointly reexamining the studies whose 
coding was in dispute. 
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Unit of Statistical Analysis 

Some studies reported more than one finding for a given 
outcome area. Such findings sometimes resulted from the use 
of more than one experimental or control group in a single 
study, and they sometimes resulted from the use of several 
subscales and subgroups to measure a single outcome. Using 
several ES s to represent results from one outcome area of 
one study seemed to be inappropriate to us because the ES s 
were usually non independent . They often came from a single 
group of subjects or from overlapping subject groups, and 
they almost always represented the effects of a single 
program implemented in a single setting. To represent a 
single outcome by several ES s would violate the assumption 
of independence necessary for many statistical tests and 
would also give undue weight to studies with multiple groups 
and multiple scales. 

The procedure that we adopted, therefore, was to 
calculate only one ES for each outcome area of each study. 
A single rule helped us to decide which ES best represented 
the study's findings. The rule was to use the ES from what 
would ordinarily be considered the most methodologically 
sound comparison when comparisons differed in methodological 
adequacy. (a) When results from both a true experimental 
comparison and a quasi-exper iment were available from the 
same study, results of the true experiment were recorded, 
(b) When results from a long and short CBE implementation 
were available, results from the longer implementation were 
used. (c/ When transfer effects of CBE were measured in 
addition to effects in the area of instruction, the direct 
effects were coded for the analysis. (d) In all other 
cases, our procedure was to use total score and total group 
results rather than subscore and subgroup results in 
calculating ES . 

Results 

Because most of the studies in the pool investigated 
effects of CBE on examination performance, we were able to 
carry out a complete statistical analysis of results in this 
area. The analysis covered both average effects and the 
relationship between effects and study features. We carried 
carried out less complete statistical analyses of other 
outcome areas because of the limited number of studies in 
these areas. 

Examination Performance 

A total of 99 of the 101 studies in our pool reported 
results from CBE and control groups on an examination given 
at the end of instruction (Table 2). In 77 of the 99 
studies, the students in the CBE class had the higher 
examination average; in 22 studies the students in the 
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conventionally taught class had the higher average. The 
difference in examination performance of CBE and control 
students was reported to be significant in 22 studies. In 
21 of the 22 cases, the significant difference favored the 
CBE class, whereas only one study favored conventional 
teaching. Overall, these box-score results favor CBE, 



Insert Table 2 about here 



The index ES provides a more exact picture of the 
degree of benefit from CBE in the typical study. The 
average ES in the 99 studies was 0.26; its standard error 
was 0.051. This average ES means that in the typical study , 
the performance of CBE students was 0.26 standard deviations 
higher than the performance of the control students. ES s 
can also be expressed in terms of pei entile scores. 
Approximately 60% of the area of the standard normal curve 
falls below a z-score of 0.26. We can conclude, therefore, 
that *he typical student in an average CBE class would 
perform at the 60th percentile on an achievement 
examination, whereas the typical student in a conventionally 
taught class would perform at the 50th percentile on the 
same examination. Put in another way, the average student 
from the CBE class would outperform 60% of the students from 
the conventional classes. 

Examination Performance and Study Features 

Although the increase in examination performance 
attributable to the computer was moderate in the typical 
study, effects varied in magnitude from study to study. The 
strongest positive result reported was an effect of 2.17 
standard deviations (Cartwright, Cartwright, & Robine, 
1972); the strongest negative result was an effect of -1.20 
standard deviations (Diem, 1982). It seemed possible that 
this variation in study outcome might be systematic, and 
further analyses were conducted to determine whether 
different types of studies were in fact producing different 
results. Three study features proved to be significantly 
related to achievement ES (?able 3). Average ES differed in 
studies that came from (a) different publication sources, 
(b) disciplines with different degrees of emphasis on 
scientific methodology; and (c) disciplines with different 
degrees of emphasis on life versus nonlife processes. 
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Publication sources . The average ES in studies found 
in professional journals was significantly higher than was 
the average effect in studies found in dissertations and 
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technical documents, F (2,96) = 3.49, £ < .05. The average 
ES in the 41 journal studies was 0.42 (SE = .08); it was 
0.16 (SE = 0.07) in the 46 dissertation studies; and it *ras 
0.11 (SE = 0.14) in the 12 technical reports. The 
difference between results found in dissertation studies and 
those found in technical documents was too small to be 
considered statistically significant, but the difference in 
results from journals and from other sources was 
significant. 

Course content . The average ES from courses in the 
hard sciences was significantly lower than the average ES 
from the soft disciplines, F (1,97) = 4.16 r e < .04. The 
average ES in the 44 studies of CBE applications in the hard 
scientific disciplines was 0.15; it was 0.35 for the 55 
studies of applications in the social sciences and 
humanities. The average ES from courses emphasizing life 
processes was also significantly higher than was the average 
ES from courses emphasizing nonlife content, F (1,97) » 
9.15, 2 < .01. The average ES in the 22 studies of life 
courses was 0.54; it was 0.18 in the 77 studies of nonlife 
courses. The coding on these two dimensions of course 
content, however, was significantly correlated, r = .29, e < 
.01. 

Other Effects 

A total of 52 of the 101 studies examined outcomes of 
CBE in areas other than* examination performance. Findings 
for these ether outcomes appear in Table 4. 
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Retention tests . Six studies examined the performance 
on follow-up examinations of CBE and conventionally taught 
classes. The follow-up interval in these studies varied 
from 2 to 10 weeks. The six studies did not seem to be 
representative of the total pool c£ studies. Whereas the 
average ES on course examinations was 0.26 for all 99 
studies, the average ES on final examinations for these six 
studies was only 0.02. It would be risky, therefore, to 
draw any general conclusions from those six studies. The 
average retention ES in the six studies was 0.18 (SE = 
0.07). 

Attitudes toward computers . Eleven studies examined 
students 1 attitudes toward computers. Contact with the 
computer in many of the studies produced positive changes in 
students' attitudes, and 7 of the 11 studies reported more 
favorable attitudes for students in the CBE class. The 
average ES in the 11 studies was 0.27 (SE = 0.16). 
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Attitudes toward instruction . Thirteen studies 
examined student ratings of the quality of instruction. 
Nine of the 13 studies found more positive attitudes in the 
CBE class; 1 study found no difference in attitudes for CBE 
and conventionally taught classes; and 3 studies found more 
negative attitudes in the CBE class. The average ES in the 
13 studies was 0.31 (SE = 0.13). 

Attitude toward subject . Fifteen studies examined the 
effects of CBE on student attitudes toward the subject 
matter that they were being taught. Only six of the 15 
studies reported that student attitudes in CBE classes were 
more positive than in conventional classes; nine studies 
found negative effects. The average ES for student 
attitudes toward instruction was -0.03 (SE = 0.07), a very 
small negative effect. 

Course completion . Twenty-one studies compared the 
numbers of students completing CBE and conventional classes. 
Eight of these 21 studies found higher completion rates in 
the CBE class; and thirteen studies found higher completion 
itates in the control class. The average h for attrition for 
these 21 studies was -0.08 (SE = 0.060), a very small effect 
favoring the control class. 

Instructional time . Fifteen studies compared the 
instructional time for students in the CBE and conventional 
classrooms. The ratio of instructional time for CBE 
students to instructional time for students studying 
conventionally was 0.66 in the average study. In other 
word*;, CBE students required only two-thirds as much 
instructional time as did students who were taught 
conventionally. The range of ratios varied from .38 to .97, 
but in no case did the conventionally taught class require 
more instructional time than the CBE class. 

Discussion 

This meta-analysis showed that college-level CBE has 
basically positive effects on students. It raised final 
examination scores in the typical study by 0.26 standard 
deviations, or from the 50th to the 60th percentile. This 
figure is very close to the average effect size of 0.25 
reported in our earlier meta-analysis of findings from 59 
studies of college level CBE (J. Kulik, Kulik, & Cohen, 
1980). The figure is also identical to the average effect 
size for CBE on achievement at the secondary level (Bangert- 
Drowns, Kulik, & Kulik, 1985), but it is smaller than the 
average effect size of 0.42 for CBE at the elementary level 
(C. Kulik, Kulik, & Bangert-Drowns , 1984). 

This analysis did not find any significant difference 
in effectiveness for different types of CBE implementations. 
CAI, CMI, and CEI programs all made small, positive 
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contributions to student learning. This result is 
strikingly different from precollege findings on CBE. In 
elementary schools, for example, CAI programs of drill and 
practice and tutorial instruction almost always produced 
good results, whereas CMI programs produced much weaker 
findings (C. Kulik et al., 1984). In high schools, both CAI 
and CMI produced positive results, but CEI programs 
contributed little to student achievement (Bangert-Drowns et 
al., 1985). At the college level, students seem to be able 
to adapt to a variety of uses of the computer in teaching. 

The relationship between study features and study 
outcomes was not strong in this meta-aualysis. Design 
features of experiments, for example, did not influence 
outcomes. Quasi-experimental studies and true experiments 
produced similar results. Experiments with controls for 
historical effects yielded the same results as experiments 
without historical controls. Such findings were not 
surprising to us. They have emerged repeatedly in meta- 
analyses of findings from educational research. 

Publication source of a study, however, was 
significantly related to study outcome. Results found in 
journal articles were clearly more positive than were 
results from dissertations and technical documents. The 
difference in effects from these different sources was not 
only highly significant, but it was also highly predictable. 
A difference between journal and dissertation results has 
been reported in numerous quantitative syntheses of research 
and evaluation findings (Bangert-Drowns et al., 1984; Glass 
et al., 1981, pp. 64-68). The relationship is one of the 
best documented findings in the meta-analytic literature. 

The factors that produce this difference, however, are 
not completely understood. A number of writers have 
attributed the difference in journal and dissertation 
findings to publication bias (e.g., Clark, in press). This 
is the purported tendency of researchers, reviewers, and 
editors to screen reports for publication on the basis of 
size and statistical significance of effects, rather than on 
the basis of study quality. Such publication bias would 
make journals an unreliable source for information about the 
effectiveness of experimental treatments. Other writers 
have noted that journal studies and other studies are 
carried out by different persons working under different 
conditions (e.g., J. Kulik, Kulik, & Bangert-Drowns, in 
press). The typical author of a jourial article, for 
example, differs from the typical dissertation writer in 
research experience, resources, professional status, and 
many other respects. If the weakness of dissertation 
results is attributable to the inexperience of dissertation 
writers, then dissertations would be a poor source for 
information on the effectiveness of treatments. 
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Strength of results was also a function of the content 
area in which the CBE evaluation was carried out. Effects 
of CBE were less clear in disciplines emphasizing hard 
science and nonlife studies; effects were clearer in 
disciplines emphasizing life studies and a softer, less 
scientifically rigorous approach; typical of disciplines 
with less clear effects are mathematics, chemistry, physics, 
and engineering. Typical of disciplines with clearer 
effects are the social sciences. A similar relationship 
between size of effect and discipline area of the study has 
been found in other quantitative syntheses of college level 
findings (C. Kulik, Kulik, & Cohen, 1980). 

The factors that produced this relationship — like the 
factors behind the relationship of publication source to 
effect size — are not yet fully understood. It is possible, 
on the one hand, that findings from social science courses 
are stronger because the quality of evaluation studies is 
better in the social sciences. Most evaluations from the 
social sciences are produced by evaluators with a strong 
background in and a professional identification with 
behavioral measurement. Other explanations of this result 
are also possible, however. Instruction in the hard, 
nonlife sciences may be more difficult to improve because 
students in these areas may already be achieving at or near 
their maximum. Or teachers in the social sciences maybe 
more discerning in their use of CBE than are teachers in the 
natural sciences. 

Another important finding in this meta-analysis was the 
reduction in instructional time associated with CBE. In 
each of the 15 studies that reported results on 
instructional time, the computer did its job quickly — on the 
average in about two-thirds the time required by 
conventional teaching methods. It is clear therefore that 
the computer can function satisfactorily in college courses 
and at the same time reduce time spent in instruction. In 
addition, computer-based teaching also had small and 
positive effects on attitudes of college students toward 
instruction. College students tended to like their courses 
somewhat more when instruction was computer-based. Finally, 
computer-based teaching had a positive effect on student 
attitudes toward the computer. 

Although CBE produced only modest effects in the 
typical evaluation study, some individual studies reported 
large effects. Included among the studies that reported^ 
unusually strong, positive effects are several in education 
and psychology: Cartwright, Cartwright, and Robine (1972); 
Green and Mink (1973); Lorber (1970); and Roll and Pasen 
(1977). Other studies that reported strong positive effects 
come from the area of music education: Humphries (1980) and 
Vaughn (1977). researchers may wish to scrutinize results of 
these atypical studies very carefully. The CBE programs 
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evaluated in these studies may point the way to better uses 
of CBE in the years ahead. 
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Table 1 

Categories Used to Describe Study Features 



Computer Use 

Type of application 

Computer-assisted instruction (CAI) — The computer 

provides (a) dr ill-and-pract ice exercises but not new 
materials, or (b) tutorial instruction that includes 
new material. 

Computer-managed instruction (CMI) — The computer 

evaluates student test performance, guides students 
to appropriate instructional resources, and keeps 
records of student progress. 

Computer-enriched instruction (CEl) — The computer (a) 
serves as a problem-solving tool, (b) generates data 
at the student's request to illustrate relationships 
in models of social or physical reality, or (c) 
executes programs developed by the student. 

Duration of instruction 

One semester or less 

More than one semester 

Author of program 

Local — Computer materials were developed locally for a 
specific setting. 

Other — Computer materials were developed for use in a 
wide variety of settings. 

Type of computer interaction 

Off-lint, 

Terminal with mainframe 
Microcomputer 
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Methodology 

Subject assignment 

Random — Subjects were randomly assigned to the 
experimental and control groups. 

Nonrandom — A quasi-experimental design was used. 

Control for instructor effects 

Same instructor — The same teacher or teachers taught 
both the experimental and control groups. 

Different instructors — Different teachers taught the 
two groups. 

Control for historical effect 

Same semester — Subjects in experimental and control 
groups were taught concurrently. 

Different semesters — Two groups were not taught 
concurrently. 

Control for test-author bias 

Commercial — A standardized test was used as the 
criterion measure for student achievement. 

Local — A locally developed tests was used as the 
criterion measure. 

Control for bias in test scoring 

Objective — Objective, machine-scorable examinations 
were used to measure student achievement, e.g., 
multiple-choice tests. 

Nonobjective — Subjective decisions had to be made in 
scoring tests, e.g., essay tests. 

Control for evaluator involvement 

Involved — The evaluator was involved in developing the 
CBE material and/or in conducting the CBE program. 

Not involved 

Field-tested computer materials 

Yes 

No 
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Settings 

Course emphasis on science 

"Hard" Science — Course emphasizes the hard sciences, 
engineering, mathematics, or agriculture. 

"Soft" discipline — Emphasis is on the social sciences, 
humanities, or education. 

Course emphasis on pure knowledge 

Pure knowledge — Course does not emphasize practical 
application of knowledge, e.g., English, chemistry, 
physiology, and psychology courses. 

Applied knowledge — Emphasis is on practical 

application, e.g., courses in mechanical engineering, 
special education, and economics. 

Course emphasis on life systems 

Life — Course emphasis is on living or organic objects 
of study, e.g., courses in physiology, special 
education, and psychology. 

Non-life systems — Emphasis is on inanimate objects, 
e.g., courses in chemistry, mechanical engineering, 
English, and economics. 

Subject ability level 

Low: Average SAT scores for freshmen entering the 

institution are below 425 or average ACT is below 20. 

Average: Average SAT between 425 and 525, or average 
ACT between 20 and 23. 

High: Average SAT between i25 and 800, or average ACT 
above 23. 



Publication history 

Year of the report 
Before 1969 
1970 - 1974 
1975 - 1979 
1980 - 1984 
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Source of study 

Technical report — Clearinghouse document , paper 
presented at a convention, etc. 

Dissertation 

Professional journal — Journal article, scholarly book, 
etc. 
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Table 2 

Major Features and Achievement Effect Sizes (ES) In 99 Studies of Computer-Based Education 



Duration 



9 

ERIC 



Study 




P 1 ace 


Course content 


Use 


in weeks 


11 


Alrd, 1974 




University of Virginia 


Mechanical 
engineering 


Management 


18 


0 


.70 


Alderman, 1978 




Northern Virginia Community 
College & Phoenix College 


Engl 1 sh & math 


Tutor 1 a 1 


18 


0 


. 15 


Allen, 1972 




Ohio State University 


Psychol ogy 


Tutor lal 


3 


0 


.07 


Anandam, Elsel & 


Kotler , 


Community College In Florida 


English composition 


Management 


12 


0 


.40 


1980 
















Anderson, 1975 




University of Illinois 


Economics 


Tutor lal 


18 


0 


. 14 


Andrews, 1974 




Florida State University 


French 


D & P 


12 


0 


.26 


Arnett, 1976 




California State College, 
Domlnguez Hills 


Account Ing 


Problem solving 


6 


0 


. 16 


Axeen, 1967 




University of Illinois 


Library science 


Tutor lal 


9 


-0 


13 


Barrozo Richards 


& 


Medgar Evans College, CUNY 


Bas 1c skills 


Tu tor 1 a 1 


<8 


0 


19 


Olsen. 1978 
















Bax ter , 1 975 




Georgl a Southwestern 
Col 1 ege 


Account 1 ng 


Problem solving 


1 1 


0 


03 


Bell, 1970 




Cornel 1 Un1vers1 ty 


Cal cuius 


Programming 


4 


0 


23 


Blckerstaff, 1977 




Kansas State University 


Math 


D & P 


3 


0. 


24 a 


Bitter. 1971 




University of Denver 


Cal cul us 


Programming 


18 


0. 


28 


Boen, 1983 




University of Arkansas 


Study ski 1 Is 


Tutor lal 


1 


0. 


92 


Boysen & Francis, 


1982 


Iowa State University 


Biomechanics 


D 8t P 


1 


0. 


62 


Broh, 1975 




State University of New 
York, Genesco 


American government 


Problem solving 


5 


0. 


21 


Byers. 1974 




University of Minnesota 


Quant i tat 1 ve 


Tutor 1 a 1 


12 


-0. 


08 
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Table 2 (continued) 
Study 

CartwMght, Cartwrlght, 

& Robine, 1972 
Caruso, 1970 
Cast 1 eberry , Montague 

& Lagowski . 1970 
Cokewood, 1980 
Coombs, 1976 
Cox, 1974 

Crawford, Montague, 

& Smith 
Culp & Lagowski, 1971 
Cunningham & Fuller, 1973 
Daellenbach, Schoenberg r 

& Wehrs, 1977 
Daughdrlll. 1978 

DeBoer, 1974 
Deloatch, 1978 
D 1 ew , 1982 

DuBoulay & Howe, 1982 
Durgin, 1979 
El linger & Frankland, 
1976 

Emery & Enger, 1972 
Fiedler, 1969 
Fr iesen, 1977 
Goodson, 1975 



Place 

Pennsylvania State Un1vers1 

University of Pittsburgh 
University of Texas 

Kean Col 1 ege 
University of Illinois 
Arizona State University 
University of Illinois 

University of Texas 
University of Nebraska 
University of Wisconsin, 

Whl tewater 
Copiah-Lincoln Junior 

Col 1 ege 
Vanderbl * 1 Un1 vers 1 ty 
Indiana University 
Florida Atlantic 

Unl versi ty 
Edi nburgh, Scot 1 and 
Black Hills State College 
University of Iowa 

St . Olaf Col lege 
Black Hawk College 
Kansas State University 
University of Houston 
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Course content 

Except 1onal 
chl ldren 
Library science 
General chemistry 

E 1 ectronlcs 
American government 
Economics 
Economics 

Cheml stry 
Phys Ics 
Economics 

Al gebra 

Calcul us 

Compensatory math 
A 1 gebra 

Math 
Math 

Geography 

Economics 
Math 
Math 
Algebra 



Ourat Ion 



Use ES 
In weeks 

Tutorial 10 2.17 

Tutorial 1 -0.32 

Tutorial 18 0.40 

Simulation 10 0.33 

Simulation 8 0.25 

Simulation 18 0.22 

Management 18 0.02 

Tutorial 18 0.32 

Tutorial 1 0.38 

Tutorial 18 0.04 

Programm 1 ng 18 0 . 09 

Programm 1 ng 18 0 . 03 

Programm i ng 18 0 . 03 

Tutorial 2 -1 .20 

Problem Solving 17 0.10 

Programming 14 0.02 

Simulation 1 -0. 14 

Simulation 1 0.43 

Programming 18 0.33 

Tutorial 1 -0.10 

Tutorial 6 0.04 



Table 2 (continued) 
Study 

Grandey, 1971 
Gray, 1973 

Green & Mink. 1973 
Hamm, 1976 

Henry & Ramsett, 1978 
Herbert, 1981 

Hofstetter, 1975 

Hollen, Bunderson, 

& Dunham, 1971 
Hololen, 1970 
Homeyer , 1970 
Hong. 1973 

Hughes, 1977 

Humphries, 1980 
Ibrahim, 1970 
Johnson, 1970 

Johnson & Plake, 1981 

Jcnes & Sorl 1e, 1976 



Place 

University of Illinois 
Oregon State University 

Macalaster College 
East Texas State 

Un1 versl ty 
University of North Dakota 
University of Wisconsin, 

Wh1 tewater 
University of Delaware 

University of Texas 

Moorhead State College 
University of Texas 
State Island Community 

College, CUNY 
Richland Community 

Col lege 
Arizona State University 
SUNY, Brockport 
University of North 

Carol ina 
University of Nebraska 

University of Illinois 
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Course content 

Chemi stry 
Operat ions 

management 
Psychology 
Counselor education 



Use 

Tutorial 
Simulat ion 

S1mu lat 1on 
Tutorial 



Durat ion 
in weeks 

3 

1 

2 
6 



ES 

0.69 
0.25 

1 .27 
0.08 



Economics 
Punctuation usage 



Management 
Tutor ial 



18 
1 



0.34 
0.35 



Ear Training 1n 

Music 
Chemi stry 



D & P 



S imul at ion 



0.69 



0.24 



Cal cuius 

Computer programming 
Account ing 



Programming 18 0. 10 

Tutorial 15 -0.21 

Problem solving 18 0.41 



Business machines 



Tutor ial 



15 



0. 17 



Music theory D & P 

Calculus Tutorial 
Computer programming D & P 



3 
2 
18 



1 .94 
0.08 
-0.22 



Advanced statistical Tutorial 
methods 

Basic science in Tutorial 
medical school 



18 



0.86 



36 0.48 
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Table 2 (continued) 
Study 

Karon, 1976 

Kelley, 1972 
Kockler. 1973 
Lang, 1974 
Larson, 1982 

Lawler, 1971 
LeCuyer, 197/ 

Lee. 1973 
Liu, 1975 

Lorber, 1970 

Mancuso, 1975 

McAdams, 1978 

Meyer & Beaton, 1974 
Mltzel, 1967 

Mltzel, 1967 

Montanelli, 1979 
Morrison & Adams, 1968 



PI ace 

Northwestern University 

University of Wisconsin 
Iowa State University 
University of Texas 
Grand Val ley State 

Col lege 
Florida State College 
University of 

Massachusetts 
University of Texas 
Western Michigan 

Un1 versl ty 
University of Athens 

University of Southern 

Mississippi 
University of Missouri, 

Rol la 
Couth Africa 
Pennsylvania State 

Un1 versl ty 
Pennsylvania State 

Un1 versl ty 
University of Illinois 

SUNY. Stony Brook 



Durat ion 

Course content Use £S 

in weeks 



Learn 1 ng 


Tutor 1al 


6 


0. 19 


dlsabll itles 








Economics 


Management 


8 


0.29 


Math 


D & P 


6 


0. 18 


Calculus 


Problem solving 


7 


0.24 


Nurs 1ng 


S 1mul at ion 


1 


0. 15 


Health education 


Management 


10 


-0.11 


Introductory math 


Programming 


18 


0.24 8 


Introductory geology 


Tutor 1al 


18 


-0.22 


General physics 


Tutor 1 a 1 


6 


0.64 


Instruct 1onal process 


Tutor 1al 


5 


1 .33 


& curriculum 








Broadcast economics 


S 1mul at 1on 


18 


-0.01 


Programming 


D & P 


13 


-0.55 


Phys 1cs 


Tutor 1 a 1 


1 


-0.02 


Engineering 


Tutoi 1al 


6 


0.28 


Speech pathology & 


Tutor 1al 


5 


-0.82 


audlology 








Computer programming 


Tutor 1al 


18 


0.24 8 


German 


Tutor 1al 


18 


-0.26 
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Table 2 (continued) 
Study 

Murphy & Appel . 1978 
Dates, 1983 

Paden, Dalgaard & Barr, 

1977 
Proctor, 1969 
Rice, 1974 

Roe & Aiken. 1976 
Rol 1 & Pasen, 1977 
Roman 1uk, 1978 

Rota. 1°32 
Saul, 1975 

SkavaMl. 1974 
Skavar 1 1 , B1rky , 

Duhrkopt, & Knight, 1976 
Smith, 1976 

Stelnkamp, 1977 
Suppes & Mornings tar, 

1969 
Swlgger, 1976 
Thompson, 1977 
T1ra, 1977 



Place 

Community College 1n 

1 1 1 1no1s 
Indiana University 
University of Illinois 

Florida State University 
Georgia State University 

University of Tennessee 

Harper Col lege 

Northern Alberta Institute 

of Technology, Canada 
Robert Morris College 
M1 ami -Dade Community 

Col 1 ege 
Ohio State University 
Ohio State University 

Orange Coast Community 

Col 1 ege 
University of Illinois 
Stanford University 

University of Iowa 
Riverside City College 
University of Missouri, 
Kansas City 
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Course content 

Cheml stry , biology, 

math & Engl 1sh 
Language arts 
Economics 



Use 
Tutorial 

Tutor 1al 
Tutor 1al 



Durat 1on 
1n weeks 
12 

18 
6 



-0.01 

0.46 
-0.04 



E ducat 1on 
Cal cuius 

Educat 1on 
Psychology 

Computer programming 



Tutor 1al 
Tutorial 

S 1mul at ion 
Management 
Tutor 1al 



2 
3 

2 
18 
2 



0.44 

0.24* 
0.67 
1 .46 
0.06 



Data processing 
Account 1 ng 



Tutorial 6 -0. 16 

Problem solving 18 0.04 



Stat 1st 1cs 
Genet 1cs 



Tutorial 
Tutor 1al 



1 1 
12 



0. 16 
0.20 



Psychology 



Management 



15 



-0.02 



Stat 1st 1cs 
Russ 1an 



S 1mu 1 at Ion 
Tutor lal 



18 
36 



0.22 
0.71 



Methods of Instruction Tutorial 

Econom 1 cs Management 

Dental Tutorial 
Classl f 1cat 1on 



2 
18 
1 



0.78 
0.22 
1 .25 
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Table 2 ( cont Inued ) 
Study 

Tol lef son, 1978 

Torop, 1975 

Tsal & PoM , 1977 
Underkof f ler, 1970 
Vaughn, 1977 
Ward & Bal lew, 1972 

Weiss, 1971 
Wolcott, 1976 
Wood, 1976 
a 



Place 

University of Kansas 

West Chester State 

Col 1 ege 
University of Santa Clara 
Winona State College 
Oregon State University 
East Texas St^te 

Un1 vers 1 ty 
New York University 
Ocean County College 
BMgham Young University 



Course content 

Educat lonal 

Measurement 
Chemistry 

Computer programming 

Math 

Mus 1c 

Set theory 

Phys 1cs 
Typewr 1 t 1ng 
Cataloging 



Use 
Management 

Tutor 1al 

Tutor 1al 
Management 
Tutor 1al 
Tutor 1al 

Management 
Tutor 1al 
Management 



Durat 1on 
1n weeks 
14 

18 

15 
9 
8 
2 

14 
15 
18 



0.52 

-0.25 

0.47 
0.33 
1 .84 
-0.86 

0.23 
-0.34 
0.20 



This study yielded a positive effect that was not statistically significant. The report did not Include enough 
detail, however, for direct calculation of ES. The ES reported here and used 1n the analysis 1s an estimated value; 
It Is the median ES in all studies of CBE that reported a statistically significant ES. 
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Computer-Based Education 

Table 3 

Means and Standard Errors of Effect Sizes (ES) for 
99 CBE Studies Classified by Study Features 



Categories 

Use of computer 
CAI 
CMI 
CEI 

Duration of Instruction 

1-4 weeks 

5-8 weeks 

9-12 weeks 

13-16 weeks 

17 or more 
Author of program 

Local 

Other 
Type of interaction 

Off-line 

Terminal w?th main frame 
Microcomputer 
Subject assignment 
Random 
Nonrandom 



ES 

N 

M SE 



58 


.26 


.08 


13 


.35 


.11 








27 


.32 


.12 


17 


.32 


.14 


12 


30 


17 


9 


.03 


.12 


34 


.24 


.06 


83 


.26 


.06 


16 


.28 


.11 


21 


.18 


.03 


73 


.27 


.06 


5 


.43 


.47 


39 


.31 


.11 


60 


.23 


.04 
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Compute r-Based Education - 38 



Table 3 ( continued ) 

Categories 

Instructors 
Same 

Different 
Semesters 
Same 

Different 
Test author bias 

Commercial test 

Local test 
Test scoring bias 

Objective 

Non objective 
Evaluator involvement 

Involved 

Not involvad 
Field tested material 

Yes 

No 

Content emphasis on 
"Hard" discipline* 

Hard 

Soft 



ES 

N 

M SE 

68 .23 .10 

31 .32 .06 

95 .26 .05 

4 .26 .21 

14 .26 .05 

85 .26 .16 

83 .26 .06 

16 .29 .09 

88 .27 .06 

11 .20 .09 

37 .31 .06 

62 .23 .10 



44 .15 .06 

55 .35 .08 
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39 



Table 3 ( continued ) 

Categories 

Content emphasis on 
"Pure" knowledge 

Pure 

Applied 
Content emphasis on 
"Life" studies** 

Life 

Nonlif e 
Ability of subjects 

Low 

Average/mixed 
High 

Nature of publication* 

Unpublished 

Dissertation 

Published 
Year of publication 

1965-1969 

1970-1974 

1975-1979 

1980-1984 



ES 

N 

M SE 



58 


.26 


.07 








22 


.54 


• 14 


77 


i ft 




29 


.17 


.10 


43 


.30 


.08 


27 


.29 


.08 


12 


.11 


.14 


46 


.16 


.07 


41 


.42 


.08 


7 


.08 


.20 


35 


.27 


.09 


45 


.25 


.06 


12 


.40 


.21 
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Table 4 

Effects in Other Outcome Areas 



Effect size (ES) 



Study 

Alrd, 1974 
Alderman, 1978 
Anandam, Else! & 

Kotter. 1980 
Anderson, 1975 
Axeen, 1967 
Baxter, 1975 
Bell, 1970 
Bickerstaff. 1977 
Bitter, 1971 
Boen, 1983 
Broch, 1975 
Byers, 1974 
CartwMght & Robine, 

1972 

Castleberry et al . , 1973 
Cokewood, 1980 
Culp & Lagowski, 1971 
Culp, Statter & 
Gilbert, 1973 
Daellenbach et al., 1977 
Daughdrill. 1978 
Deboer, 1974 



Instruct ional 
time (Ratio 
of X:C) 



.69 
.85 



.53 



67 



Course 
complet 1on 

-.05 
-.68 



- . 14 



.04 



00 



00 



08 
09 



.08 



12 



Retent 1on 
at 

f ol 1 ow-up 



.06 



Attitude Attitude 

toward toward 
computer 1 nstruct 1on 

-.36 

. 18 
.57 
.37 

-.01 
.22 

-. 12 
18 
.64 



.72 



Att 1 tude 
toward 
subject 



13 



.23 



.31 



.09 



.26 
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Table 4 ( continued ) 



Effect size (ES) 



Study 

Deloatch, 1978 
D1em, 1982 
Durgln, 1979 
Fiedler, 1969 
FMesen, 1977 
Grary f 1973 
Green & M1nk f 1973 
Hollen et al . . 1971 
Hololen, 1970 
Homeyer, 1970 
Hong, 1978 
Hughes, 1977 
Ibrahim, 1970 
KavaMI, 1974 
Kelley, 1972 
Kockler, 1973 
Larson, 1982 
Lawler, 1971 
Lee, 1973 
Locker, 1970 
Meyer & 

Beaton, 1974 
MontanelH, 1979 



ERLC 



Instructional 
time (Ratio 
of X:C) 



97 



.48 



.57 



.82 

.50 
64 



Course 
compl et 1on 

-.46 
.04 



.30 



17 



.32 



Retention 
at 

f ol low-up 



. 10 



.08 



18 



Attitude Attitude 

toward toward 
computer 1 ns truct 1 on 



-.75 



1 .05 
.54 



.06 



.99 



-.26 



.00 



Att1 tude 
toward 
subject 
.61 

-.37 

-.29 

-.34 

-.08 



-. 10 
.24 



.05 



.09 
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Table 4 ( cont Inued ) 



Study 

Morrison & 

Adams, 1968 
Murphy & 

Appel, 1978 
Proctor, 1969 
Rol 1 & 

Pasen, 1977 
Roman luk, 1978 
Roter f 1982 
Saul, 1975 
Smith, 1976 
Stelnkamp, 1977 
Suppes & 

Mornlngstar, 1969 
Thompson, 1977 
Tollefson, 1978 
Ward & 

Ballew, 1972 
Wolcott, 1976 



Effect size (ES) 

I ns t rue 1 1 ona 1 Cou r se 

Retention Attitude Attitude Attitude 

time (Ratio completion 

at toward toward toward 

of X:C) (h) 

follow-up computer Instruction subject 



.68 



.07 



.53 



19 



.02 



.49 



.80 



.02 
.22 



-.46 



.96 



.07 
1 .03 



.72 
.04 



.51 



38 
.64 



10 
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